Artificial intelligence in COVID-19

The COVID-19 pandemic has taken the world by storm, placing healthcare systems around the globe under immense pressure. The exceptional circumstance has made the scientific community turn to artificial intelligence (AI), with hopes that AI techniques can be used in all aspects of combating the pandemic, whether it is in using AI to uncover sequences in the genomic code of the severe acute respiratory syndrome coronavirus (SARS-CoV-2) virus for the purposes of developing therapeutics, such as antivirals, antibodies, or vaccines, or using AI to provide (near-) instantaneous clinical diagnosis techniques by way of analysis of chest X-ray (CXR) images, computed tomography (CT) scans or other useful modalities, or using AI for as a tool for mass population testing by analyzing patient audio recordings. In this chapter, we survey the AI research literature with respect to applications for COVID-19 and showcase and critique notable state of the art approaches. © 2023 Elsevier Inc. All rights reserved.

A Temporal-oriented Broadcast ResNet for COVID-19 Detection

Detecting COVID-19 from audio signals, such as breathing and coughing, can be used as a fast and efficient pre-testing method to reduce the virus transmission. Due to the promising results of deep learning networks in modelling time sequences, we present a temporal-oriented broadcasting residual learning method that achieves efficient computation and high accuracy with a small model size. Based on the EfficientNet architecture, our novel network, named Temporaloriented ResNet (TorNet), constitutes of a broadcasting learning block. The network obtains useful audio-temporal features and higher level embeddings effectively with much less computation than Recurrent Neural Networks (RNNs), typically used to model temporal information. TorNet achieves 72.2% Unweighted Average Recall (UAR) on the INTERPSEECH 2021 Computational Paralinguistics Challenge COVID-19 cough Sub-Challenge, by this showing competitive results with a higher computational efficiency than other state-of-the-art alternatives. © 2022 IEEE.

Example-based Explanations with Adversarial Attacks for Respiratory Sound Analysis

Respiratory sound classification is an important tool for remote screening of respiratory-related diseases such as pneumonia, asthma, and COVID-19. To facilitate the interpretability of classification results, especially ones based on deep learning, many explanation methods have been proposed using prototypes. However, existing explanation techniques often assume that the data is non-biased and the prediction results can be explained by a set of prototypical examples. In this work, we develop a unified example-based explanation method for selecting both representative data (prototypes) and outliers (criticisms). In particular, we propose a novel application of adversarial attacks to generate an explanation spectrum of data instances via an iterative fast gradient sign method. Such unified explanation can avoid over-generalisation and bias by allowing human experts to assess the model mistakes case by case. We performed a wide range of quantitative and qualitative evaluations to show that our approach generates effective and understandable explanation and is robust with many deep learning models. Copyright © 2022 ISCA.

Multi-Type Outer Product-Based Fusion of Respiratory Sounds for Detecting COVID-19

This work presents an outer product-based approach to fuse the embedded representations learnt from the spectrograms of cough, breath, and speech samples for the automatic detection of COVID-19. To extract deep learnt representations from the spectrograms, we compare the performance of specific Convolutional Neural Networks (CNNs) trained from scratch and ResNet18-based CNNs fine-tuned for the task at hand. Furthermore, we investigate whether the patients' sex and the use of contextual attention mechanisms are beneficial. Our experiments use the dataset released as part of the Second Diagnosing COVID-19 using Acoustics (DiCOVA) Challenge. The results suggest the suitability of fusing breath and speech information to detect COVID-19. An Area Under the Curve (AUC) of 84.06 % is obtained on the test partition when using specific CNNs trained from scratch with contextual attention mechanisms. When using ResNet18-based CNNs for feature extraction, the baseline model scores the highest performance with an AUC of 84.26 %. Copyright © 2022 ISCA.

Recognising Covid-19 from Coughing using Ensembles of SVMs and LSTMs with Handcrafted and Deep Audio Features

As the Covid-19 pandemic continues, digital health solutions can provide valuable insights and assist in diagnosis and prevention. Since the disease affects the respiratory system, it is hypothesised that sound formation is changed, and thus, an infection can be automatically recognised through audio analysis. We present an ensemble learning approach used in our entry to Track 1 of the DiCOVA 2021 Challenge, which aims at binary classification of Covid-19 infection on a crowd-sourced dataset of 1 040 cough sounds. Our system is based on a combination of hand-crafted features for paralinguistics with deep feature extraction from spectrograms using pre-trained CNNs. We extract features both at segment level and with a sliding window approach, and process them with SVMs and LSTMs, respectively. We then perform least-squares weighted late fusion of our classifiers. Our system surpasses the challenge baseline, with a ROC-AUC on the test set of 78.18 %.

Coughing-based Recognition of Covid-19 with Spatial Attentive ConvLSTM Recurrent Neural Networks

The rapid emergence of COVID-19 has become a major public health threat around the world. Although early detection is crucial to reduce its spread, the existing diagnostic methods are still insufficient in bringing the pandemic under control. Thus, more sophisticated systems, able to easily identify the infection from a larger variety of symptoms, such as cough, are urgently needed. Deep learning models can indeed convey numerous signal features relevant to fight against the disease;yet, the performance of state-of-the-art approaches is still severely restricted by the feature information loss typically due to the high number of layers. To mitigate this phenomenon, identifying the most relevant feature areas by drawing into attention mechanisms becomes essential. In this paper, we introduce Spatial Attentive ConvLSTM-RNN (SACRNN), a novel algorithm that is using Convolutional Long-Short Term Memory Recurrent Neural Networks with embedded attention that has the ability to identify the most valuable features. The promising results achieved by the fusion between the proposed model and a conventional Attentive Convolutional Recurrent Neural Network, on the automatic recognition of COVID-19 coughing (73.2 % of Unweighted Average Recall) show the great potential of the presented approach in developing efficient solutions to defeat the pandemic.

The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primates

The INTERSPEECH 2021 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the COVID-19 Cough and COVID-19 Speech Sub-Challenges, a binary classification on COVID-19 infection has to be made based on coughing sounds and speech;in the Escalation Sub-Challenge, a three-way assessment of the level of escalation in a dialogue is featured;and in the Primates Sub-Challenge, four species vs background need to be classified. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' COMPARE and BoAW features as well as deep unsupervised representation learning using the AUDEEP toolkit, and deep feature extraction from pre-trained CNNs using the DEEP SPECTRUM toolkit;in addition, we add deep end-to-end sequential modelling, and partially linguistic analysis.

COVID-19's Impact on Mental Health-The Hour of Computational Aid?

Welcome to the fourth issue of IEEE Transactions on Computational Social Systems (TCSS) in 2022. First, we have some exciting news to share. In late June, Clarivate updated the Impact Factor of all journals which are indexed by Web of Science. According to the Journal Citation Reports, the 2021 Journal Impact Factor of IEEE TCSS was 4.727. Many thanks to all for your great effort and support. After the usual introduction of our 25 regular articles, we would like to discuss the topic of "COVID-19's Impact on Mental Health-The Hour of Computational Aid?"

Learning complementary representations via attention-based ensemble learning for cough-based COVID-19 recognition

Coughs sounds have shown promising as-potential marker for distinguishing COVID individuals from non-COVID ones. In this paper, we propose an attention-based ensemble learning approach to learn complementary representations from cough samples. Unlike most traditional schemes such as mere maxing or averaging, the proposed approach fairly considers the contribution of the representation generated by each single model. The attention mechanism is further investigated at the feature level and the decision level. Evaluated on the Track-1 test set of the DiCOVA challenge 2021, the experimental results demonstrate that the proposed feature-level attention-based ensemble learning achieves the best performance (Area Under Curve, AUC: 77.96%), resulting in an 8.05% improvement over the challenge baseline. ©

CONVOLUATIONAL TRANSFORMER WITH ADAPTIVE POSITION EMBEDDING FOR COVID-19 DETECTION FROM COUGH SOUNDS

Covid-19 has caused a huge health crisis worldwide in the past two years. Although an early detection of the virus through nucleic acid screening can considerably reduce its spread, the efficiency of this diagnostic process is limited by its complexity and costs. Hence, an effective and inexpensive way to early detect Covid-19 is still needed. Considering that the cough of an infected person contains a large amount of information, we propose an algorithm for the automatic recognition of Covid-19 from cough signals. Our approach generates static log-Mel spectrograms with deltas and delta-deltas from the cough signal and subsequently extracts feature maps through a Convolutional Neural Network (CNN). Following the advances on transformers in the realm of deep learning, our proposed architecture exploits a novel adaptive position embedding structure which can learn the position information of the features from the CNN output. This make the transformer structure rapidly lock the attention feature location by overlaying with the CNN output, which yields better classification. The efficiency of the proposed architecture is shown by the improvement, w. r. t. the baseline, of our experimental results on the INTERPSEECH 2021 Computational Paralinguistics Challenge CCS (Coughing Sub Challenge) database, which reached 72.6 % UAR (Unweighted Average Recall). © 2022 IEEE

AN OVERVIEW OF THE FIRST ICASSP SPECIAL SESSION ON COMPUTER AUDITION FOR HEALTHCARE

Audio has been increasingly used as a novel digital phenotype that carries important information of the subject's health status. We can find tremendous efforts given to this young and promising field, i. e., computer audition for healthcare (CA4H), whereas the application scenarios have not been fully studied as compared to its counterpart in medical areas, computer vision. To this end, the first special session held at ICASSP 2020 was dedicated to the topic. In this overview paper, we at first summarise the invited high-quality contributions from leading scientists from a multi- disciplinary background. Then, we provide a detailed grouping of the contributions to several scenarios such as body sound analysis (e. g., heart sound), human speech analysis (e. g., stress detection), and artificial hearing technologies (e. g., cochlear implants). In addition to the collected works, we will compare them with other recent studies within the topic. Finally, we conclude the limitations and perspectives of the current stage. It is interesting and encouraging to find that the state-of-the-art machine learning and audio signal processing techniques have been successfully applied in the health domain, e. g., to fight with the global challenges of COVID-19 and ageing population. © 2022 IEEE

Editorial: Intelligent Signal Analysis for Contagious Virus Diseases

COVID-19 infection-s recent outbreak triggered by the SARS-CoV-2 Corona virus had already led to more than two million reported infected individuals when we first addressed the community by our call - by now, the number sadly rose to roughly half a billion cases worldwide. The outbreak of COIVD-19 has also re-shaped and accelerated the scientific publication landscape in no time. One can observe a massive uprise in interest in work related to the topic of highly contagious virus diseases and potential contributions of digital health including intelligent signal processing. In addition, most publishers have reacted in one or the other way to the crises such as by opening up to pre-prints, waiving publication fees for COVID-19-related research, providing search functions and tools for COVID-19 research, and many more. Here, we gathered 13 carefully selected novel contributions across signal types such as audio, speech, image, video, or symbolic information, as well as their multimodal combination for application in the risk assessment, diagnosis, and monitoring of contagious virus diseases. © 2007-2012 IEEE.

Speaking Corona? Human and Machine Recognition of COVID-19 from Voice

With the COVID-19 pandemic, several research teams have reported successful advances in automated recognition of COVID-19 by voice. Resulting voice-based screening tools for COVID-19 could support large-scale testing efforts. While capabilities of machines on this task are progressing, we approach the so far unexplored aspect whether human raters can distinguish COVID-19 positive and negative tested speakers from voice samples, and compare their performance to a machine learning baseline. To account for the challenging symptom similarity between COVID-19 and other respiratory diseases, we use a carefully balanced dataset of voice samples, in which COVID-19 positive and negative tested speakers are matched by their symptoms alongside COVID-19 negative speakers without symptoms. Both human raters and the machine struggle to reliably identify COVID-19 positive speakers in our dataset. These results indicate that particular attention should be paid to the distribution of symptoms across all speakers of a dataset when assessing the capabilities of existing systems. The identification of acoustic aspects of COVID-19-related symptom manifestations might be the key for a reliable voice-based COVID-19 detection in the future by both trained human raters and machine learning models. Copyright ©2021 ISCA.

Cough-based COVID-19 detection with contextual attention convolutional neural networks and gender information

The aim of this contribution is to automatically detect COVID- 19 patients by analysing the acoustic information embedded in coughs. COVID-19 affects the respiratory system, and, consequently, respiratory-related signals have the potential to contain salient information for the task at hand. We focus on analysing the spectrogram representations of cough samples with the aim to investigate whether COVID-19 alters the frequency content of these signals. Furthermore, this work also assesses the impact of gender in the automatic detection of COVID-19. To extract deep-learnt representations of the spectrograms, we compare the performance of a cough-specific, and a Resnet18 pre-trained Convolutional Neural Network (CNN). Additionally, our approach explores the use of contextual attention, so the model can learn to highlight the most relevant deep-learnt features extracted by the CNN. We conduct our experiments on the dataset released for the Cough Sound Track of the DICOVA 2021 Challenge. The best performance on the test set is obtained using the Resnet18 pre-trained CNN with contextual attention, which scored an Area Under the Curve (AUC) of 70.91% at 80% sensitivity. Copyright © 2021 ISCA.

The DiCOVA 2021 challenge - An encoder-decoder approach for COVID-19 recognition from coughing audio

This paper presents the automatic recognition of COVID-19 from coughing. In particular, it describes our contribution to the DiCOVA challenge - Track 1, which addresses such cough sound analysis for COVID-19 detection. Pathologically, the effects of a COVID-19 infection on the respiratory system and on breathing patterns are known. We demonstrate the use of breathing patterns of the cough audio signal in identifying the COVID-19 status. Breathing patterns of the cough audio signal are derived using a model trained with the subset of the UCL Speech Breath Monitoring (UCL-SBM) database. This database provides speech recordings of the participants while their breathing values are captured by a respiratory belt. We use an encoder-decoder architecture. The encoder encodes the audio signal into breathing patterns and the decoder decodes the COVID-19 status for the corresponding breathing patterns using an attention mechanism. The encoder uses a pre-trained model which predicts breathing patterns from the speech signal, and transfers the learned patterns to cough audio signals. With this architecture, we achieve an AUC of 64:42% on the evaluation set of Track 1. Copyright © 2021 ISCA.

Detecting COVID-19 from breathing and coughing sounds using deep neural networks

The COVID-19 pandemic has affected the world unevenly;while industrial economies have been able to produce the tests necessary to track the spread of the virus and mostly avoided complete lockdowns, developing countries have faced issues with testing capacity. In this paper, we explore the usage of deep learning models as a ubiquitous, low-cost, pre-testing method for detecting COVID-19 from audio recordings of breathing or coughing taken with mobile devices or via the web. We adapt an ensemble of Convolutional Neural Networks that utilise raw breathing and coughing audio and spectrograms to classify if a speaker is infected with COVID-19 or not. The different models are obtained via automatic hyperparameter tuning using Bayesian Optimisation combined with HyperBand. The proposed method outperforms a traditional baseline approach by a large margin. Ultimately, it achieves an Unweighted Average Recall (UAR) of 74.9%, or an Area Under ROC Curve (AUC) of 80.7% by ensembling neural networks, considering the best test set result across breathing and coughing in a strictly subject independent manner. In isolation, breathing sounds thereby appear slightly better suited than coughing ones (76.1% vs 73.7% UAR). © 2021 IEEE.

Recent Advances in Computer Audition for Diagnosing COVID-19: An Overview

Computer audition (CA) has been demonstrated to be efficient in healthcare domains for speech-affecting disorders (e. g., autism spectrum, depression, or Parkinson's disease) and body sound-affecting abnormalities (e. g., abnormal bowel sounds, heart murmurs, or snore sounds). Nevertheless, CA has been underestimated in the considered data-driven technologies for fighting the COVID-19 pandemic caused by the SARS-CoV-2 coronavirus. In this light, summarise the most recent advances in CA for COVID-19 speech and/or sound analysis. While the milestones achieved are encouraging, there are yet not any solid conclusions that can be made. This comes mostly, as data is still sparse, often not sufficiently validated and lacking in systematic comparison with related diseases that affect the respiratory system. In particular, CA-based methods cannot be a standalone screening tool for SARS-CoV-2. We hope this brief overview can provide a good guidance and attract more attention from a broader artificial intelligence community.

A comparison of acoustic and linguistics methodologies for Alzheimer's dementia recognition

In the light of the current COVID-19 pandemic, the need for remote digital health assessment tools is greater than ever. This statement is especially pertinent for elderly and vulnerable populations. In this regard, the INTERSPEECH 2020 Alzheimer's Dementia Recognition through Spontaneous Speech (ADReSS) Challenge offers competitors the opportunity to develop speech and language-based systems for the task of Alzheimer's Dementia (AD) recognition. The challenge data consists of speech recordings and their transcripts, the work presented herein is an assessment of different contemporary approaches on these modalities. Specifically, we compared a hierarchical neural network with an attention mechanism trained on linguistic features with three acoustic-based systems: (i) Bag-of-Audio-Words (BoAW) quantising different low-level descriptors, (ii) a Siamese Network trained on log-Mel spectrograms, and (iii) a Convolutional Neural Network (CNN) end-to-end system trained on raw waveforms. Key results indicate the strength of the linguistic approach over the acoustics systems. Our strongest test-set result was achieved using a late fusion combination of BoAW, End-to-End CNN, and hierarchical-attention networks, which outperformed the challenge baseline in both the classification and regression tasks. Copyright © 2020 ISCA

An early study on intelligent analysis of speech under COVID-19: Severity, sleep quality, fatigue, and anxiety

The COVID-19 outbreak was announced as a global pandemic by the World Health Organisation in March 2020 and has affected a growing number of people in the past few weeks. In this context, advanced artificial intelligence techniques are brought to the fore in responding to fight against and reduce the impact of this global health crisis. In this study, we focus on developing some potential use-cases of intelligent speech analysis for COVID-19 diagnosed patients. In particular, by analysing speech recordings from these patients, we construct audio-only-based models to automatically categorise the health state of patients from four aspects, including the severity of illness, sleep quality, fatigue, and anxiety. For this purpose, two established acoustic feature sets and support vector machines are utilised. Our experiments show that an average accuracy of.69 obtained estimating the severity of illness, which is derived from the number of days in hospitalisation. We hope that this study can foster an extremely fast, low-cost, and convenient way to automatically detect the COVID-19 disease. © 2020 ISCA